A FREE ANALOGUE OF SHANNON'S PROBLEM ON MONOTONICITY 

OF ENTROPY 



DIMITRI SHLYAKHTENKO. 



Abstract. We prove a free probability analog of a result of |ABBNfl4j . In particualar 
we prove that ii Xi,X2, ■ ■ ■ are freely independent identically distributed random variables, 
then the function 

"^H — — ) 

is monotone increasing for all n. 



1. Introduction. 
Let X be a random variable with law /i and let 

X(X) = j Jlog\s-t\dfx{s)dfx{t) + ^ + hog27i. 

This quantity, called free entropy, was discovered by Voiculescu in |Voi93| and plays the role 
of entropy in his free probability theory (see e.g. |Voin2| for a survey). Free entropy has nice 
behavior with respect to freely independent random variables. Amazingly, its behavior is in 
many instances parallel to the behavior of classical entropy, if one replaces the classical notion 
of independence by that of free independence. For example, for variables with variance 1, 
the free entropy is maximized by the semicircular law — t^X[-^ ^]{'t)dt (which plays 

the role of the Gaussian law in the free central limit theorem) |Voi97j . Similarly, one has the 
free analogue of the entropy power inequality |SV96| . 

In analogy to the classical case, Voiculescu developed an infinitesimal theory for x with 
respect to free Brownian motion |Voi98| . If is a random variable with the semicircular law, 
then 

$(x) = ^ x{x + Vis) 

at t=o 

is called (by analogy with the classical setting) the free Fisher information of X. If X has 
law /i with density dfi{x) = f{x)dx, then 

$(X) = ^ 1 {f{x)fdx. 

Free entropy can be recovered from the free Fisher information using the formula 

(1) x{X) = \j - <I>(X + VtS)^ + ^ log 2vre 



(a similar formula, which involves $(ve^X + ^/l — e "^^S) can be derived by a change of 
variables) . 
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The free Fisher information also has many properties analogous to those of classical Fisher 
information. These include the free analogs of the Cramer-Rao inequality and also of the 
Stam inequality. 

In |ABBNn4| . the authors have solved an old problem going back to Shannon on mono- 
tonicity of entropy (see (HW49, Sta59, Lie78| V In particular, they have proved that if 
Xi, X2, . . . is a sequence of iid random variables, then the classical entropy H of their central 
limit sums is increasing: 

^/ Xi + --- + X„ \ ^^^Xi + --- + X„+i 



Their proof relied on a new variational characterization of Fisher information (Theorem 4 in 
|ARRN04| V 

The main purpose of this note is to derive the free analog of this statement. Namely, if 
Xi,X2, . . . are identically distributed and freely independent, then the inequality 

( Xi + ■ ■ ■ + X„ \ / Xi + ■ ■ ■ + X„+i 
X 7= ^ X 



^ J \ Vn + 1 

holds for all values of n. 

The proof in the free case is in many instances a repetition of the argument of |ABBN04] . 
taken with minimal modifications; we provide the full arguments for completeness. Indeed, 
once we obtain a free analog of equation (4) of [ABBNOl] in LemmaElthe rest of the argument 
is effectively the same as in |7rBBNn4| . with H replaced by x- However, we do not know 
of a free analog of the variational characterization (Theorem 4 in [ABBNQ4J), and so we 
take a slightly different route to derive the analog of (4) in |ABBNfl 4j. going back to the 
definition of the free Fisher information using the "conjugate variables" (which are the free 
analogs of the classical score function), and is in its spirit similar to the proof of the free 
Stam inequality )Voi98j . 

It is curious to note that our proof can also be used, with appropriate modifications, also 
in the classical case, thus giving a somewhat shorter argument for the results of jABBNOj] . 
avoiding the use of (Theorem 4 in |ABBNn4] ). We outline this approach in Sectional It 
would be interesting if this approach could be used to give a speedier proof of the variational 
principle in [ABBN04J. 

As in fABBNn4l| . we prove results which are stronger than monotonicity of entropy. These 
include generalizations to many summands of the free Stam inequality and the free entropy 
power inequality. These results are stated and proved in Section [21 



2. Monotonicity of Free Entropy. 

Let (X, Y) G (M, r) be two self-adjoint random variables. Following |Voi98| . we denote by 
dx:Y the derivation from the algebra generated by X and Y into L^(M) (8)L^(M) determined 
by dx:Y{X) = 1®!, dx-.viy) = and the Leibniz rule. Then the conjugate variable J(X : Y) 
(which is the free analog of the classical score function) is the (unique, if it exists) vector in 
L^(M, r) satisfying 

( J(X : r), P) = (1 ® 1, dx:Y{P)) =r® r{dx:Y{P*)), 
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for any non-commuting polynomial P in X and Y. One write J{X) for J{X : 0). By 
definition, the free Fisher information is given by 

$(X) = ||J(X)||i.(^,). 

We caution the reader familiar with |ABBNn4) that in that paper the symbol J denotes the 
classical analog of $ and not of the conjugate variable J. 

Lemma 1. Let Xi, . . . , X„ G (M, r) be self-adjoint random variables. Let ai, . . . , a„+i G M. 
Assume that for some j , J(^i-^j caXi : Xj) exists. Then 

n+l 

jC^ttiXi) = Ej^,(^n+^i^^^^)(J(^aiXi : Xj)). 

1=1 ij^j 

Proof. Let f(t) = t"^ he a monomial in one variable. Let Y = o-iXi, Yj = ^i-^i^ 

Zj = ajXj, N = W*{Y). Then 

n+l n+l 

(/(J]a,X,),E^(J(J]a,X, = (/( J] a,X,), J( J] a,X, : X,)) 

i=l i^j i=l i^j 

= r(g) r{dY^^^^ aa.:xJiJ2 ^^^^ + "J^J')) 

= T®T{dy,.z,{Y, + Z,r) 
m 

fc=i 

= r®r(9y/(F)). 

Since En{J(^-_^j a-iXi : Xj)) G L'^ {W* {Yl^=i cnXij), the claimed equality follows. □ 

We get the following corollary: 
Lemma 2. Let Xi, . . . , Xn+i be freely independent. Let (ai, . . . , a^+i) G M"+^ be an (n + l)- 



tuple satisfying '^o.'j = Let bi, . . . , bn+i G M 5e so that Yl^=i ^jy^~^'j~^- Then 

n+l n+l / N 

(2) $(5^ a,X,) <nJ2 b'j^ , • 

Proof. Clearly, if OjXj) = +oo for some j, there is nothing to prove. Hence we 

assume that aiXi) is finite for all j. 

Let N = W^*(Er=i^«i^i)- According to Lemmad 



n+l 



J(5^a,X,) = E^,(J(5^a,X, : X,)). 

i=l i^j 



Hence 
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n+1 n+1 n+1 

i=l j=l i=l 

n+1 



n+1 ^ 

j=l \ 1^ ~ 



Since -^Aris a contraction on we deduce that 



'JN 

n+1 n+1 



1=1 j=i Jl — ttj i-^j 



By freeness, J{-j^= Y^i^j (^i^i '■ ^j) = ^( ^i^i) and hence if we let 

= bjJ{ = QiXi), 

then 

n+1 n+1 

$Ea.X,)< II 5^011^- 

1=1 j=i 

Now let : M = W*{Xi, . . . , X^) ^ Mj = W*{Xi, . . . , Xj, . . . , X^) be the conditional 
expectation. Then Ej : L?'{M) —>■ L'^{M) are projections and moreover Ej form a commuting 
family. Indeed, because of the freeness assumptions, we may write 

M = W*{Xi)*---*W*{Xj)*---*W*{Xn) 
Ej = id * ■ ■ ■ * t\iv*(^Xj) * ■ ■ ■ * id. 

Hence if i < j, 

Ei o Ej = id* ■■■ * T|v(/.(Xi) * id * ■ ■ ■ * id * r|vy(Xj) * id * ■ ■ ■ * id = o 

In particular, note that Ei o ■ ■ ■ o E^ = t. Since t(^j) = (because ^j is up to a multiple a 
conjugate variable), we deduce that conditions of Lemma 5 on p. 6 of [ .ABBNOi] are satisfied. 
We apply this lemma to conclude that 

n+1 n+1 n+1 

$(5^a,X,)<||5^0ll2<MEll^^- 



^3 \\2)- 

i=l j=l j=l 



Recalling that 



1 - a] i^j 
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we conclude that 



n+l n+1 rt+1 / 

i=i 3=1 j=i \^l-a- i^j 



as claimed. □ 

We can now deduce a many- variable version of the free Stam inequality |Voi98| : 
Theorem 1. Let Xi, . . . , X^+i be free random variables. Then 



n+l 

n 

> 



n+l ^ 



Proof. Let aj = {n + 1) Then 1/wl - = ^/{n + l)/n and hence ^ implies that for 



any bj with ^ bj = >y {n + 1) /n, we have 

71+1 n+l 

*(5:A-,)<«E;r^''?*(E-^')- 

i=l j=l i^j 

Hence if we are given Xj with ^ Aj = 1, we could take bj = \j^J{n + l)/n and hence deduce 
that 

n+l n+l 

(3) $(5^X,)<n5^A^^$(J]X0. 

i=l j=l i^j 

Now let C = Z]=l HJ2^^jXiy^ and let 

A, - ^ • 

Then Q becomes 

n+l n+l 

1=1 j=i 
Recalling the definition of C gives 

n+l -n+l 

i=l j=l i^j 

which clearly implies the statement of the Theorem. □ 

Theorem 2. Let Xi, . . . , Xn+i be free random variables and let (ai, . . . , a„+i) E S"' be a unit 
vector. Then 



n+l n+l ^ _ o 



(4) X{J2^^X^)>J2^-^X 



n 

i=i j=i 
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Proof. Set 6j = ^yl — a| in Lemma El and apply the Lemma to xj*^ = Xj + \/tSj. We then 
get 



n+1 



i=l i 



n 



1 - a'j i^j 



Notice that since '^af = 1, the law of a-iX^^^ is the same as the law of aiXi + 

y/iS', where S' = ^-j^j ^ semicircular variable, free from J2^=i o-i^i- Similarly, 



since a? 



;i - a]) 



we get that the law of 



^j^, ttiXf^ is the same as the law 



of {-^j=^^i-Lj o-iXi) + \/tS'^^\ where S^^^ = -^j=^^i-LjCijSj is a semicircular free from 

ajXj. Hence using the formula ^ and integrating with respect to t, we deduce the 
desired inequality (jH). □ 

Corollary 1. Let Xi, . . . be identically distributed free random variables. Then 



X 



Xi + --- + Xr, 



< 



X 



Xi + ■ ■ ■ + Xn+1 



Vn + 1 



Proof. Using Theorem [2l with aj = 1/ \/n + 1 gives us 



n+1 



i=l 7 = 1 



n+1 1 ]_ 

n+1 



1 



J=l 

n+1 



n 



X 



1 



^ n+1 



j=i V 



since Xi, . . . , X„ have the same law. 

One can also get a free analogue of the entropy power inequality for many summands: 
Theorem 3. Let Xi, . . . , be free random variables. Then 



□ 



exp 



n+1 



i=l 



^ n+1 



i=i 



Proof Let E = exp(2x(Er=/ ^0) and Ej = exp(2x(Ei^j ^i))- Note that if X and Y are 
free, then using the triangular change of variables formula |Voi94j and monotonicity of free 
entropy, one gets: 

X(X + Y) + x{Y) > x{X + Y,Y) = x(X, Y) = x(X) + x{Y), 

so that x(X + F) > x{X)- Hence E > Ej for all j. Hence if for some j, Ej > ^ X]r=i 
desired inequality E > would follow. Thus we may assume that Ej < jiY17=i -^i- 
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Let then Xj = Ej/ Y17=i -^«) thus Xj < 1/n for all j. Let aj = \/l — nXj; applying (jlj) gives 
the inequality 

n+1 n+1 

(5) ^(E^^)^E^^-^(7^E^^) 

2=1 j=i V i i^j 

Using the fact that 



2 

iytj ij^j i=l 

n+1 



© b 



ecomes 



n 

i=l 



n+1 n+1 ^ ^ n+1 ^ ^ n+1 



i=l j=l i=l i=l 

Multiplying by two and exponentiating gives 



^ n+1 

n ^-^ 

1=1 



which is the desired inequality. 



□ 



3. Some Remarks on the Classical Case. 



The argument in the free case followed almost word for word the argument of jABBNOj] in 
the classical case with one exception: we used Lemma[T]to prove Lemma [2l (which is the free 
analog of equation (4) in pVBBN04j). Once this analog of (4) in [ABBNOl) was established, 
the argument in the free case became entirely parallel to the classical case. 

We now point out that an analog of Lemma [T] also holds in the classical case, and hence 
equation (4) in [ABBNOl] can be derived this way (at least for variables that have finite 
moments of all orders) , avoiding the use of the variational characterization of classical Fisher 
information (Theorem 4 in lABBNOi] ). This approach seems to be slightly shorter and may 
shed more light at the variational characterization mentioned above. 

In the remainder of the section we'll recall some facts about the classical Fisher information 
and indicate how the analog of Lemma Q] can be proved. In order to avoid confusion with 
the notation in the free case, we shall use F to denote Fisher information. 

Let X, Yi, . . . , K„ be classical real-valued random variables, and let uj be the probability 
measure on M""^^ describing their joint law. Recall that the score function 
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is the (unique, if it exists) element satisfying 

(j(X:yi,...,y„),/(x,i/i,...,2/„)) = (i,|^). 

In other words, j{X : Yi, . . . ,Yn) = (^)* 1 if one considers d/dx to be a densely defined 
operator on L'^{uj) with domain of definition consisting of polynomials in the coordinates 
x,yi, . . . , Un- Algebraically, d/dx is determined by the fact that it is a derivation and also by 
{d/dx)x = 1, {d/dx)yk = for all /c = 1, . . . , n. When this algebraic definition is emphasized, 
we'll denote d/dx by dx:yi,...,y„. 

In the case that n = 0, one can easily check that if du{x) = f{x)dx, then 

AX) ^ L 

and hence the Fisher information is given by 

Lemma 3. Let Oi, . . . , a„+i G R. Assume that for some k, aiXi : X^) exists. Then 

n+1 

i=l i^k 

Proof. Let f{t) = t"* be a monomial in one variable. Let Y = (^t^i^ ^fc = J2i=/=k^i-^i^ 

Zk = akXk,N = W*{Y). Then 

n+1 n+1 

{fQ2a^X,),EMJ2<^^^^■^k))) = {f (J^ a,X,) , J {J^ a.X,: X^)) 

i=l i^k i=l i^k 

= {f{Yk + Zk),j{Yk:Zk)) 
= {dy,..zJiY, + Z,),l) 

m 

= Y.{{Y, + Zy-' {dy,..z,{Yu + Z,)) {Y, + Zr-\ 1) 

r=l 

= {f\Y)A). 

Since EN{j{J2i=/=k^i-^i '■ -^k)) G L'^(W*{Yl^=i(^i^i))i the claimed equality follows. □ 

Finally, it is routine to check that j{X : Y) = j{X) in the case that X and Y are classically 
independent. Thus the argument in Lemma O applies verbatim to yield the classical analog 
of and thus of equation (4) in jARRNn4| . 
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